TagC: a Language with Computation Distribution Directives for Expressing Parallelism

In this paper, we present TagC, a new language based on C for distributing parallel and/or pipelined computations. TagC simpliies the programming of distributed memory machines by providing the programmer with a global name space and a mechanism for specifying computation distribution. The main advantage of our paradigm is that it provides a unique framework for expressing both functional parallelism and data parallelism. 1 Background Parallel computers are becoming more and more accessible to scientists and programmers. However, they are not likely to be widely accepted until they are easy to program. To obtain the full performance of a distributed-memory parallel machine, the programmer is forced to manually distribute code and data, and manage the resulting communications by explicitly adding send and receive messages. Recently, new high-level programming languages were developped that make use of a global name space to facilitate the writing of programs. In this approach, the currently dominant programming model for distributed-memory machines is data-parallel paradigm with user-directed data distribution. With the owner-computes rule, the compiler derive the corresponding computation distribution. For example, this approach has been adopted in Vienna Fortran, Fortran-D, and High Performance Fortran 6, 13, 12]. In our approach, the user speciies the computation distribution rather than the data distribution. The later is derived from the former either at compile time or at runtime time. Hence, the user has more exibility to express and organize parallelism; both functional and data parallelism can be naturally expressed. Computation distribution as a programming model has been utilized on shared-memory parallel systems. See for example 4]. Yet, it is applicable to distributed-memory machines as well, as demonstrated here. Additional motivation on this paradigm can be found in 15]. In this paper, we present TagC, a new language based on C for distributing parallel and/or pipelined computations, and its implementation. It was developed for typical distributed-memory MIMD machines, the Alex Informatique AVX Series 1 and 2 machines 2]. With TagC and its compiler, the programmer can directly concentrate on high-level algorithm design and performance issues. The TagC language is an extension to C but the concept is in fact directly applicable to other imperative languages such as Fortran. In this section, we brieey illustrate the features of TagC by comparing two versions of a code written in C and TagC respectively. The program, that generates a Mandelbrot image, consists of a main program and three functions. See Figure 1. …